Getting back up to speed.
Getting back up to speed.
Completed. Recorded prelim slides, now on Drive. Need to give to Ben.
Done demo, just need prelim slides.
Workflow stopped working middle of the day.
Written the "script" for it, just need to check the poller is updating correctly (i.e. the bodge is working..).
Also need some prelim slides that outline the architecture, why we want to use tokens over X509 etc.
OIDC redirect PR still failing.
PR (5261) for mongo to be added to CI (+ pymongo reqm and docker-compose file for external metadata providers).
Autotests work something like:
tools/run_autotests.py - starting point (outside of containers)etc/docker/test/matrix - edit matrix for what distro/python vers you want to testtools/test/build_images - on subsequent runs, may want to comment ln116 to avoid rebuilds if not requiredtools/test/run_tests.py - this is the script that runs CI related commands inside the containerstools/test/before_script.py - this docker runs the necessary containersDepending on whether you're running syntax checks (which do not require httpd) or "all", need alter the corresponding run function (run_test_directly & run_with_httpd respectively). For the former, you need to comment out the call to the run function on ln246 and instead print this command to the terminal so it can be manually invoked. For the latter, you can add something like:
import time
time.sleep(36000)on ln306 which keeps the container running.
It's then possible to exec inside the container, init the database:
tools/run_tests_docker.sh -air
and then run a test, e.g.:
tools/pytest.sh -v --tb=short test_multi_vo.py -k "test_auth_oidc"
Dockerfiles used for testing (Centos, Fedora) are kept in /etc/docker/test.
Wrote tests. Getting Mayaank's feedback on CI.
Fixed whoami issue (404) by adding:
proxy_set_header X-Requested-Host $scheme://$host/rucio-dev;to the nginx route.
See rucio's flaskapi/v1/accounts.py for why (redirect was only using the account name as the redirect location because X-Requested-Host, and consequently frontend, was not set)
Got a working version (minus tests) of the mongo plugin. Should be aware that without the new filtering engine, it only works for equality and AND expressions.
Still need to write unit tests, but requires a dev environment mongodb!
Would be cool to hook this up for IVOA - I&P?
Raised PRs for OIDC stuff and request history.
Got poller working with tokens.
Started looking at Rucio PRs. Looks like i've made a mess of some of these branches.
Created a new clone with only the requests branch work in (rucio). Trying to get this pushed through now.
Should then create a new repo (with config), migrate the json stuff across, add the mongodb stuff (manually?) and look at the minor changes following oidc.
Sick leave.
This now works. Required some bug fixes in rucio codebase & change of requested scope and audience.
Got this working - was a storage side problem + the fact that the RSE had root and gsiftp enabled & it was gfal stating the file (or attempting to) using these protocols instead of tokens.
Ops meeting.
Lots of debugging.
oidc-gen needed "device" grant type on escape-rucio-analysis as it's headless (can't do via browser).
Rucio token workflow - conveyor submitter needs idpsecrets in /etc/idpsecrets and oidc_audience set to whatever is being used in the rest of the system (default 'rucio' but currently the WLCG .any URL).
Simon F. replied, he's added the audience. Can also remove requirement for wlcg.groups claim if necessary. Suspect we don't want to do this as the information for the groups is contained in here! (finer grained authz). Confirmed that having "rucio" as audience now works.
Updated ska-rucio-client container to allow oidc env vars.
Got CNAF STORM working also, but required the same as yesterday (re: audience claim). At the moment we've set expected_audience in rucio server to wlcg any URL. Will need to change this back. Works with davix-put and rucio upload.
Emailed FTS about subsequent transfer failures
Made a lot of headway here with Rohini's help.
Managed to get certificates working. Needed the withcodes/ directory in grid-security on the rucio-head machine (alternatively, can grab this from any scientific linux machine on IRIS).
Rucio upload almost works. Needed wlcg.groups claim. Need to allow "rucio" as expected audience at storage level. Emailed Simon F at Imperial.
Started process of making our own STORM storage. Server up and running but not tested.
SRC prototypes meeting.
Looking at trying to get davix-ls working with ssl certificate.
Looking at trying to get Rucio upload working.
Finished annual review admin. Started looking at verifying Imperial token stoage.
Added scopes storage:create and storage:modify to ska-rucio-auth client on INAF indigo iam.
Installed davix:
eng@ubuntu:/tmp$ sudo apt install davixSpawned the OIDC agent to get a token:
eval $(oidc-agent)
export AT=`oidc-token ska-rucio-auth`Couldn't get SSL certificate to verify, so tried with insecure:
eng@ubuntu:/tmp$ davix-put --insecure --capath /etc/grid-security-certificates -H "Authorization: Bearer ${AT}" testfile https://gfe02.grid.hep.ph.ic.ac.uk:2880/pnfs/hep.ph.ic.ac.uk/data/escape/ska/testfile
eng@ubuntu:/tmp$ davix-ls --insecure -l --capath /etc/grid-security-certificates -H "Authorization: Bearer ${AT}" https://gfe02.grid.hep.ph.ic.ac.uk:2880/pnfs/hep.ph.ic.ac.uk/data/escape/ska
-rwxrwxrwx 0 5 2022-02-01 17:07:06 testfileNeed to remove those storage scopes (presumably this is done by fts through Rucio?).
Also need to contact Mihai re: what Rizart said about FTS
it's mainly Mihai from the FTS team at CERN, we need to ask him if the pilot instance
is configured to accept ESCAPE IAM issued tokens (which as I remember from before the
holidays, it is), and then to ask if these tokens would need some specific scopes or
an audience claim "fts" for example